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Abstract 


University websites and online portals are the primary means through which potential students 
and other stakeholders find important information about an institution. University websites are 
essential to these organizations’ marketing and communication efforts. In this paper, focus has 
been put on the need to complement these websites with the use of an AI Chatbot (UniBot) in 
order to serve more efficiently. This study aims at performing an extensive literature survey on 
intelligent conversational agents and the feasibility of applying them in enhancing online 
communication in universities. The study utilizes an iterative — incremental methodology to aid 
in design and development of UniBot, using AIML (Artificial Intelligent Markup Language) 
Pattern matching algorithm on the Pandorabot (AIAAS) platform, to generate high quality 
training data, with which, the agents Natural Language Understanding (NLU) model is trained. 
The study also provides for training and testing the agent using data which is acquired from 
Online Communication, University Website department at Kenyatta University. 
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1. Introduction 


According to Melisa Perich (2015), a lot of research work has been done on Artificially 
Intelligent Systems to support education. Elis, Allen and Tuson (2010) argue that with the 
development of Artificial Intelligence, computer systems can complete or perform tasks that 
would require human intelligence at a much larger scale than we could on our own. AI has been 
applied in fields like speech recognition, visual perception, natural language processing and 
decision making. Artificially Intelligent Conversational Agents or Chatbots have being utilized in 
a number of ways. Companies like Paypal, Facebook and Tesla are increasingly relying on 
Artificially Intelligent Bots. Google has also developed Google Assistant which is a 
virtual agent that can engage users in two-way conversations. From these advancements we can 
clearly see that there is a growth in Artificially Intelligent Conversational Agents or Chatbots 
adoption rate. Modern commercial chatbots offer more advanced development environments 
enabling the design and development of intelligent conversational agents with complex and goal 
driven behavior. 
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2. Literature review 


The development of conversational agents enabled human beings to interact with 
computers by use of natural language processing (NLP). According to Atwell and Shawar, (2007), 
with the growth in pervasive computing and technologies, there developed an ardent requirement 
to interact with computers in the same way communication is done with people, by using natural 
language. According to (Zadrozny et al. 2000), by allowing users to express themselves directly 
and in a natural way, i.e., by speaking, typing, and pointing, a more sophisticated HCI can be 
achieved. Milan Van Eeuwen (2016) defines a chatbot as an overall term for describing concepts 
like chatter bots, virtual agents, and conversational agents. Desaulniers (2016) defined a chatbots 
as an interactive messaging application powered by artificial intelligence (AI). In addition to 
Desaulniers definition, Schlicht (2016) describes a chatbot as a service, powered by rules and 
sometimes artificial intelligence that you interact with using a chat interface. A chatbot is defined 
in various ways but they all describe the same phenomenon. The definitions mostly differ in 
mentioning to what extent a chatbot is driven by artificial intelligence. With more progress in 
research, Milan van Eeuwen (2017) describes a chatbot as an intelligent application that interacts 
with user in natural language through a chat and can be used effectively to serve a commercial 
purpose. According to microsoft.com (2018), Microsoft Cortana has some services like weather 
forecast, calendars, reading outlook emails, giving time estimations for travel, giving directions, 
and integrating with OneNote to show users’ their notes. According to Google Assistant (2018), 
Google Assistant is an extension of the basic OK Google functionality that allows users to conduct 
search and control their mobile devices through voice commands. Assistant is programmed into 
Google phones, Android OS and will be integrated into some cars. According to amazon.com 
(2018) (retrieved from https://developer.amazon.com/alexa), Alexa can access the weather, 
connect to radio and television stations, and has partnerships with a number of services, including: 
JustEat, Uber, FitBit, The Telegraph, Spotify and Nest, among others. These services can be 
accessed with the Alexa interface. 


3. Evaluation and results 
3.1 Approach 


Black-box approach which is qualitative assessment was chosen to evaluate UniBot. 
Black-box approach focuses on the output generated in response to the given input without 
concern with the internal processing. This approach suits the nature of UniBot well because the 
core function of UniBot is to deliver appropriate answers to users in real time in order to overcome 
some of the problems associated with the current online communication systems as stated in 
Chapter 1. Black-box approach has been used by other researchers in question-answering system 
whether in open domain (Nyberg & Mitamura, 2002) or restricted domain (Diekema et al., 2004) 
as well as for conversational bots (Goh et al., 2007; Preez et al., 2009). Turing test which has high 
regard in this field is also a black-box approach (Floridi, 2003). 


3.2 Dataset 


Datasets are required to examine the responses generated by UniBot in black-box 
evaluation. For this purpose, one hundred questions are used. 


3.3 Dataset’s results 


From the results shown in Table1i, Uni-bot was accurately able to score 47 points out 
of 100 points. The scores are grouped according to the frequency so that trend can be observed in 
Table 5.5 and Figure 1. 
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Table 1. Frequency of points 





Figure 1. Scores frequency chart 


From the table and chart shown in Table 1 and Figure 1 respectively, it can be seen that 
the majority of the dataset results fall at 4-point score. This means that Uni-bot was able to provide 
correct answers at most of the time during the test and some even with creative answers. Out of 
the hundred questions, 5 percent of the answers are in categories of 0-point to 2-point. This shows 
that Uni-bot was able to response appropriately by at least giving some relevant answers. 
However, the existence of 0-point results means that there are still some questions ended up with 
completely irrelevant responses. In this case, Uni-bot still needs to be further improved in the 
future. 


3.4 Summary 


This chapter saw the evaluation of UniBot being conducted using one hundred 
datasets compiled from online chat communication logs from KU Online Communication 
Department. The evaluation is done using the black-box approach which completely relied on the 
outputs generated and in this case are UniBot’s responses. From the results gathered, it can be 
seen that UniBot was able to provide correct answers at most of the time and only small percentage 
of 5 percent having unrelated responses. 
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In terms of the objectives of this project, the evaluation showed that UniBot 
demonstrated the success of achieving the three objectives stated in Chapter 1. 


3.5 Conclusion 


This research has demonstrated a conversational agent for a university website users 
implemented using AIML. The evaluation results have shown that UniBot has great potential to 
interact with end users. Even though the knowledge in this study is limited to a single university, 
the solutions is customizable to fit any university setting. The advantages of UniBot such as being 
able to provide 24-hour service that can serve different time zones, able to have knowledge in 
multiple domains, and one conversational bot can be shared by multiple sites simultaneously have 
outweighed its existing limitations which are mainly due to external factors. 
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